1. Field of the Invention
The present invention relates to a method and apparatus for synthesizing a plurality of frames, sampled from a moving image, to obtain a synthesized frame having a higher resolution than the sampled frames. The present invention also relates to a program that causes a computer to execute the method for synthesizing moving images.
2. Description of the Related Art
With the recent spread of digital video cameras, it has become possible to handle moving images in units of single frames thereof. When these frames of the moving image are to be output as prints, it is necessary to heighten their resolutions, to improve the image qualities thereof. Therefore, methods for sampling a plurality of frames from a moving image, then generating a single synthesized frame having higher resolution than the plurality of frames, have been proposed (for example, in Japanese Unexamined Patent Publication No. 2000-354244). These methods derive motion vectors among the plurality of sampled frames. Then, the methods calculate signal values to be inserted among pixels, based on the motion vectors, when generating the synthesized image from the plurality of frames. In the method disclosed in Japanese Unexamined Patent Publication No. 2000-354244, each of the sampled frames is divided into a plurality of blocks. Then, perpendicular coordinate coefficients that correspond among the frames are calculated. Thereafter, data regarding high frequency components within the perpendicular coordinate coefficients are synthesized with low frequency components within other blocks, to calculate pixel values to be inserted. Therefore, necessary data is not reduced, and a synthesized frame having high image quality can be obtained. In addition, this method calculates motion vectors at resolutions finer than the distances between pixels. Therefore, movement between frames is accurately compensated for, enabling the obtainment of synthesized frames having still higher image quality.
Another method for synthesizing frames of a moving image has been proposed, as disclosed in: Y. Nakazawa, T. Komatsu, and T. Saito, “Obtainment of Finely Detailed Digital Images by Integrating Frames”, Journal of the Television Academic Society, 1995, Vol. 49, No. 3 p 299-308. This method designates one of a plurality of frames as a reference frame. A reference patch, having a plurality of rectangular regions therein, is provided in the reference frame. A second patch, which is the same as the reference patch, is provided in a second frame other than the reference frame. The second patch is moved and/or deformed so that the image within the second patch matches the image within the reference patch. Then, correspondent relationships between pixels within the second patch and pixels within the reference patch are estimated, based on the moved and/or deformed second patch and the reference patch. Thereby, a plurality of frames are accurately synthesized.
In the method disclosed by Nakazawa et al., correspondent relationships among the reference frame and the other frames are estimated. Following this estimation, the reference frame and the other frames are assigned to an integrated image, having a necessary resolution. Thereby, a finely detailed synthesized frame is obtained.
There are cases in which a subject included in a frame, or included in a local portion of a frame, moves greatly or at extremely high speed among frames. In these cases, movement and/or deformation the patches in the method disclosed by Nakazawa et al. may not be able to follow such movement. In the case that the movement and/or deformation of the patches cannot follow the movement and/or deformation of the subject, the entirety of a synthesized frame may be blurred, or the subject therein, which exhibits great movement, may be blurred. Thus, there is a problem that a synthesized frame having high image quality cannot be obtained. In these cases, the movement and/or deformation of the patches can be made to follow the movement and/or deformation of the subject, by dividing the patches into a greater number of rectangular regions. However, if the number of rectangular regions is excessively increased, then each of the rectangular regions becomes extremely small, thereby causing great difficulty in following the movement and/or deformation of the subject.